1. Field of the Invention
The present invention relates to a camera device to which multiple differently focused images can be imaged at the same time, and an image processing method in which the desired image is generated from multiple differently focused images.
2. Related Art Statement
As a conventional example of the image for generating desired image or picture from multiple images, for example, there is an image-processing device based on the region division. In this conventional image processing method, for example, multiple different focused images are prepared, the focused region is judged respectively in respective images, the desired image is generated by performing a series of processing by which a fixed visual effect is given to each region by dividing the region of the multiple image based on the judgment result. In that case, it is general to use the image processing program which describes the procedure to process the above region decision, the region division, and the visual effect successively,
Moreover, as a conventional example of the camera device used to image the multiple images used for the above image processing method, there is for example a usual imaging lens control type camera device 51 shown in FIG. 2. This camera device 51 is constituted as a camera device, which has one optical system. In order to obtain multiple images which are imaged by different focusing for the same scene (for example, scene 52 of FIG. 2) by using conventional camera device 51. It is necessary to perform imaging of multiple times by changing focusing.
That is, when n kinds of images are imaged according to different focusing by using conventional camera device 51 shown in FIG. 2, a zoom lens 53 is controlled by the manual operation, or by servo lens controller 54 provided to outside of the camera, first of all, after the focusing of a zoom lens 53 is controlled to focus on the first depth, the first image is acquired, next, after the focusing of the zoom lens 53 is controlled to focus on the second depth, the second image is acquired, and similarly after the focusing of the zoom lens 53 is controlled to focus on n depth, the nth image is acquired. Thus, the focusing and the imaging of n times are needed to acquire the image focused on n kinds of depth. Moreover, the imaged picture is forwarded from CCD not shown in the figure to a workstation 55, and will be supplied to the image processing.
Since the above conventional image processing method uses the decision condition xe2x80x9cFocused regionxe2x80x9d, when the region which has the even brightness value in the imaging subject scene and the region with the depth change are existent, the judgment precision of the region decision cannot be secured enough about those regions. Therefore, the above application range of the conventional image processing method is limited to a sharpening of the image etc. according to the integrated focused regions, so that the extension into more functional image processing is extremely difficult, such as the focus blur is arbitrarily adjusted every region and a stereoscopic image etc. is given, thereby generating the pseudo disparity.
Moreover, as long as the above conventional camera device in which the system of controlling the zoom lens from the outside is adopted, it is necessary to repeat the imaging by one imaging section every depth to be focused. Multiple images with different focusing cannot fundamentally be imaged to the dynamically changed imaging subject at the same time, so that it is not possible to use for the imaging of moving scene, and thus the application is limited only to the still image.
In addition, the zoom lens generally used as an imaging lens has an extremely complex control structure using the multiple lenses itself, if focusing is adjusted from the outside, it is not avoided that the optical center of an internal lens group which is the effective view position (main point) changes into back and forth direction of an optical axis. Therefore, when focusing is adjusted, three dimensional conditions will also be added, for example, a phenomenon is caused since an object in the back plain covered with a front object in the picture imaged in a certain condition, appears newly in the scene because of moved view point and in another image and thus imaged. As a result, the requirement xe2x80x9cThe imaged subject (object etc.) is same, except for the magnification change at the entire image among images in which the same scene is imaged, even in the change of focusingxe2x80x9d can not be met, so that as for the image generated by the image processing, deterioration in picture quality which originates in the above phenomenon is not avoided by using the imaged picture or image.
The first object of the present invention is to provide a camera device in which multiple differently focused images can be imaged at the same time.
The second object of the present invention is to provide an image processing method in which the desired image is generated from multiple differently focused images.
To this first end, according to the present invention, there is provided a camera device for multiple pictures imaging types comprising an imaging lens mechanism; a prism means having an incident surface for receiving a light emanated from the imaging lens mechanism, and n output surfaces for emitting distributed light by dividing the incident light into n multiple equal parts (n=2,3 . . . ), respectively; n imaging sections for imaging the distributed light emitted from the n output surfaces of the prism means, respectively; a focusing control mechanism for adjusting a positional relation between the n output surfaces and the corresponding imaging sections, respectively; and a synchronizing means for synchronizing a timing of the imaging by the n imaging sections; whereby multiple images may be imaged by different focussing, simultaneously.
To this second end, according to the present invention, there is provided an image processing method using multiple images comprising steps of: imaging multiple differently focused images Im (m=1, . . . , n); deriving a following identical equation based on the superposition models realized between the different multiple focusing images Im and desired images I;
f(T1, . . . , Tk, T11, . . . , T1n, T21, . . . , T2n, Tm1, . . . , Tmk, Tn1, . . . , Tnn, I, I1 , . . . , In)=0xe2x80x83xe2x80x83(1)
where, n=2, 3, . . . , k=1, . . . , n and m=1, . . . , n, Tmk is a function showing convolution according to blur function decided from camera property and camera parameter, and Tk is a function specified by a user showing a convolution according to blur function in the image and showing an amount of parallel movement corresponding to disparity;
obtaining a solution by substituting multiple images Im and the functions Tmk and Tk of different focusing into the identical equation;
performing an image processing for generating desired image I based on this solution; and
selectively generating a desired image I by controlling the function Tk to obtain one of (a) any one of various images which contain a full focused image in which a full region of the image is sharpened, (b) an arbitrary focused image in which sharpness was adjusted every depth, (c) an image with disparity for binocular vision in which disparity is given, (d) an arbitrary focused image with disparity for binocular vision in which disparity is given to the arbitrary focused image, (e) a partially emphasized image in which only a specific region is emphasized, or (f) a partially extracted image in which only a specific region is extracted.
According to the present invention, as a preprocessing of encoding, a blooming control is preformed on depth direction every different region.
According to the present invention, the light emitted from the imaging lens mechanism enters into an incident plane of the prism means, thereby becoming multiple n (n=2, 3, . . . ) distributed lights divided equally. These multiple distributed lights enamate from n output planes of the prism means, respectively, thereby being imaged on n imaging sections, respectively. In that case, since the positional relation between the n output planes and the corresponding imaging sections is adjusted respectively with the focusing control mechanism, and the imaging according to the n imaging sections is synchronously performed by the synchronous means, so that multiple differently focused images can be imaged at the same time.
The different multiple images of focusing obtained by this imaging meet the requirement xe2x80x9cThe imaged subject (object etc.) is same, except for the magnification change at the entire image among images in which the same scene is imaged, even in the change of focusingxe2x80x9d, and can be used to process the image to aim at the generation or the like of a stereoscopic image. Moreover, multiple images of different focusing according to the above imaging are obtained at the same time, so that multiple images can correspond dynamically also in case of changing the subject (moving image etc.).
According to the present invention, each of multiple images of different focusing and the desired image to be generated are shown as superposition Im (m 1, . . . , n) and 1, so that the identical equation (1) approved immediately between the multiple images and the desired image and based on the superposition model, is led, and thus desired image I can be generated with the image processing. In this case, the element depending on the region division which is a common element to the multiple images and the desired image, is deleted from the above identical equation, so that the malfunction, for which the judgement precision of the region judgement as the above conventional example cannot be secured, is not caused, and thus the malfunction, from which the range of application of the image processing method is limited to the sharpening of the image etc., is not caused either.
According to the present invention, since the above identical equation, by which the relation approved immediately between multiple images and desired images is shown, can be applied to various images, so that a control of the function Tk, as a desired image I, generates selectively any one of various images which contain a full focused image in which a full region of the image is sharpened, an arbitrary focused image in which sharpness was adjusted every depth, an image with disparity for binocular vision in which disparity is given, an arbitrary focused image with disparity for binocular vision in which disparity is given to the arbitrary focused image, a partially emphasized image in which only specific region is emphasized, and a partially extracted image in which only a specific region is extracted. (For example, if the function Tk is adjusted to achieve the effect of the high pass filter, the partial emphasized image to emphasize only a specific region is obtained, if the function Tk is adjusted to achieve the effect of the phase shift filter, the image with disparity for the binocular vision with given disparity is obtained, and if the function Tk is adjusted to achieve the effect of full zero filter on an unnecessary region, the partial extraction image, in which only a specific region is extracted, is obtained.)
According to the present invention, it is preferable to perform an adaptive image compression (for example, low rate encoding) in each region, since the adjustment, by which the focus blur is emphasized or is controlled in the direction of depth in each different region, is performed as a preprocessing of encoding.